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Abstract 

The period of 2010 to 2014 marked a relatively stable stage in the evolving 
quality assurance system for Ontario postsecondary education, particularly 
following massive changes after 2000. The current system consists of three 
frameworks overseen respectively by three quality assurance agencies — the 
Ontario Universities Council on Quality Assurance, the Ontario College Quality 
Assurance Service, and the Postsecondary Education Quality Assessment Board, 
each serving different purposes. This paper uses van Vught and Westerheijden’s 
(1994) general model of quality assessment and Jeliazkova and Westerheijden’s 
(2002) Phase Model of quality assurance systems development as heuristic 
tools for the analysis. The following four areas are discussed: formalized 
quality mechanisms with diversity; the arms-length tripartite relationship with 
a distinction between self-regulation and government regulation; mixed phases 
of quality assurance development and the maturing process; and gaps in the 
system. It is argued that within fifteen years, a comprehensive and relatively 
mature quality assurance system with diverse but rigorous mechanisms has been 
developed for Ontario postsecondary education although a few deficiencies exist, 
including the continued lack of coordination within the system. The evolution 
of the system appears to reflect a path trajectory of governance structure 
development of the Ontario postsecondary education system as well as be a result 
of interactions between the local and the global. 
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1. Introduction 

The shifting landscape of postsecondary education in the past few decades 
has heightened the attention to quality-related issues. Factors that have lent 
prominence to quality assurance issues include the expansion and diversification 
of postsecondary education, internationalization, cuts in public funding, 
privatization of higher education, new approaches to public administration, and 
international competition (Brennan & Shah, 2000; El-Khawas, 2007; El-Khawas, 
DePietro-Jurand, & Holm-Nielsen, 1998). Quality in postsecondary education has 
become a central concern and there is a global trend toward the establishment of 
rigorous quality assurance systems in countries across the world. 

Under the Canadian constitution, education is under the jurisdiction of the 
government of each of the ten provinces and three territories. Each of them has its 
own postsecondary education system as well as quality assurance frameworks and 
mechanisms. This highly decentralized governance structure has made Canada 
quite a distinctive case in terms of quality assurance (Weinrib & Jones, 2014). 
There is lack of consistency in the definitions^ 1 of the term quality assurance and 
in the approaches to quality assurance. 

Traditionally, major quality assurance mechanisms used in Canada include 
legislation; affiliation and federation with public universities; external and 
internal reviews; provincial/territorial registration/licensing; and professional 
accreditation (Canadian Information Centre for International Credentials [CICIC], 
2009). The uncoordinated quality assurance mechanisms have been labeled 
“patchwork” (Oldford, 2006). It is widely acknowledged as a complex challenge 
to use a unified mechanism across the postsecondary sector for quality assurance 
and to assess quality assurance mechanisms for different types of institutions 
and those mechanisms in various jurisdictions (Canadian Council on Learning 
[CCL], 2009; Klassen, 2012; Oldford, 2006). To address this gap, a couple of 
pan-Canadian initiatives have taken place in the past decade, including the release 
of the Canadian Degree Qualifications Framework (Council of Ministers of 
Education, Canada [CMEC], 2007) and the creation of the pan-Canadian Quality 
Assurance Framework (CICIC, 2012). However, these coordinating documents 


[1] Here are two definitions used by two pan-Canadian organizations: “Quality assurance refers to the 
achievement of educational program standards established by institutions, professional organizations, 
government, and/or standard-setting bodies established by government” (CICIC, 2009); “Quality assurance 
refers to the criteria and processes that are employed in reviews of institutions and/or programs to determine 
whether standards set for postsecondary curriculum, outcomes, and input are being met and maintained, 
and whether they encourage continuous improvement in the quality of higher education” (CMEC, n.d.). 
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do not change the scenario where each province and territory has its own 
independent and self-sustained system of quality assurance for postsecondary 
education. 

This paper focuses on the quality assurance system for postsecondary 
education in Ontario, the most populous province in Canada and also the one 
with the largest postsecondary education system. In the following sections, I 
will first provide an overview of the evolution and the key components of the 
three frameworks of the Ontario quality assurance system and then evaluate the 
features of the system in light of two conceptual models of quality assurance 
(Jeliazkova & Westerheijden, 2002; van Vught & Westerheijden, 1994) on the 
basis of several data sources. I will also provide some interpretations as to why a 
system with those features has come into being, and argue that the emergence of 
the Ontario quality assurance system was influenced by certain local and global 
factors. 

The history of external quality appraisal for Ontario postsecondary 
education began in the 1960s. Since then, different, uncoordinated quality-related 
mechanisms were put in place for academic programs at Ontario universities 
and colleges. The past fifteen years have been particularly eventful. Three 
quality assurance agencies that are external to postsecondary institutions have 
emerged: the Ontario Universities Council on Quality Assurance (also known as 
the Quality Council) as mandated by the Quality Assurance Framework (QAF) 
for Ontario public universities and their programs; the Ontario College Quality 
Assurance Service (OCQAS) to assure the quality of programs offered by public 
colleges; and the Postsecondary Education Quality Assessment Board (PEQAB) 
for assessment of degree-granting programs offered outside Ontario public 
universities. By 2010, an external quality assurance system driven by the work 
of the three agencies had come into being. The system remained stable till the 
end of 2014. In 2015, the dynamics within the system have been shifting again 
as the OCQAS framework starts to implement a new College Quality Assurance 
Accreditation Process, which instills another major change within the whole 
quality assurance system in Ontario. 

Given this context, it is meaningful to examine the quality assurance 
practices during the stable period of 2010 to 2014 and analyze the features 
therein. This endeavour will not only help the stakeholders in Ontario to develop 
a better understanding of the status of quality assurance for postsecondary 
education within the province but also demonstrate to readers from other parts 
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of the world how a well-balanced quality assurance system has evolved in a 
democratic society such as Canada. The analysis will hopefully stimulate more 
ideas on how to enhance quality assurance mechanisms in other jurisdictions. 

2. Methodology 

The description and evaluation of the quality assurance system for Ontario 
postsecondary education in this paper are informed by the following data sources 
relating to its three components - the QAF framework, the OCQAS framework, 
and the PEQAB framework. 

(1) Publicly available documents published on the websites of the three quality 
assurance agencies; 

(2) Interview data I collected from a total of 14 key informants who are currently, 
or were previously, involved with the development or implementation of the 
quality assurance processes under the three frameworks; and 

(3) Archives provided by two of the interviewees regarding how QAF and 
OCQAS frameworks were first developed. 

The arguments in this paper have also drawn upon my observations of the 
case studies I conducted at seven Ontario postsecondary institutions regarding 
their practices in implementing those frameworks (Liu, 2015) and at three 
universities regarding change management during the transitional period of 
implementing the QAF (Liu & Liu, 2015). Data triangulation (Patton, 2002) was 
applied to verify information from different sources. 

In addition, the analysis and evaluation in this paper is also made in light of 
van Vught and Westerheijden’s (1994) general model of quality assessment for 
postsecondary education and Jeliazkova and Westerheijden’s (2002) Phase Model 
of quality assurance systems development. Both recognize the distinction between 
the intrinsic and extrinsic qualities of postsecondary education — the inherent 
values of the pursuit of knowledge and the search for truth, and the necessity to 
respond to the changing demands from society. The general model posits that a 
well-established quality assurance mechanism is characterized by a system-level 
agency operating independently from the government; academics’ ownership 
and self-evaluation of the process; peer reviews with site visits; reporting for 
improvement purposes rather than for judgment; and no direct, rigid connections 
between quality review results and funding decisions. 

The Phase Model represents the contingencies in the evolution of 
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quality assurance systems. It defines four types of quality assurance problems, 
which range from “serious doubts about educational standards” (Phase 1) 
to “need to stimulate sustainable quality assurance in institutions” (Phase 4) 
(Jeliazkova & Westerheijden, 2002, p. 435), with an increasing trust of the 
public in postsecondary institutions. To address those problems, the role of 
quality assurance at Phase 1 is to identify sub-standard educational programs 
whereas at Phase 4, it splits between improvement based on self-regulation and 
public accountability. The nature of the external review is summative through 
accreditation at Phase 1 and split between an audit report to the institution 
and verification of the data to be incorporated in public databases. The model 
demonstrates the evolving relations between postsecondary education and the 
government from a “one-to-one control mechanism” at the lower phase of quality 
assurance development to “more complex and presumably more effective forms 
of accountability, where concepts as academic excellence and autonomy take on a 
new meaning” at an advanced phase (Jeliazkova & Westerheijden, 2002, p. 437). 
As a quality assurance system moves toward a higher phase, its focus shifts from 
the external dynamics to the internal dynamics. 

3. An Overview of the Ontario Quality Assurance System 

Currently, Ontario’s postsecondary education system consists of 21 publicly 
assisted universities, 24 publicly assisted colleges (including two French- 
speaking ones), 17 private universities (almost all have a religious affiliation), 
and over 250 registered private career colleges. The public university and college 
sectors have different roles and distinct governance structures running parallel 
to each other and constituting the mainstream of the system (Jones, 1997). This 
historically formulated binary structure makes it reasonable to establish separate 
quality assurance mechanisms for the two sectors. 

3.1 The QAF Framework for the University Sector 

The Quality Assurance Framework (QAF) represents the quality assurance 
policy for the Ontario university sector. It is important to note that the QAF 
is not a government-created document; rather, it was the output of a two-year 
development process (2008 - 2009) under the leadership of a special task force, 
which worked closely with the academic vice-presidents of Ontario universities 
and consulted with the Ontario government responsible for postsecondary 
education (i.e., the Ontario Ministry of Training, Colleges and Universities, 
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or OMTCU). The QAF was unanimously approved by the Ontario Council of 
Academic Vice-Presidents (OCAV) in February 2010. In the same year, a quality 
council was established by the OCAV to replace its predecessor — the Ontario 
Council on Graduate Studies (OCGS). Previously, quality assurance for public 
Ontario universities was administered under a binary arrangement: appraisals of 
graduate programs were conducted by OCGS whereas undergraduate programs 
were subject to audits by the Undergraduate Program Review Audit Committee [2] 
(UPRAC) under the direction of OCAV A significant change as a result of the 
QAF is the unification of the previously separate mechanisms under the OCGS 
and the UPRAC. 

The implementation of QAF is overseen by a quality agency under the 
Council of Ontario Universities, known as the Quality Council. The Quality 
Council undertakes two major responsibilities: approving new undergraduate 
and graduate programs, as well as auditing each university’s internal quality 
assurance process, known as the Institutional Quality Assurance Process (IQAP), 
on an eight-year cycle. Those two functions are implemented by an appraisal 
committee and an audit committee. Although it makes periodic reports to the 
government, the Quality Council is responsible to the OCAV and the Council of 
Ontario Universities, which are essentially the universities themselves, without 
being monitored by the government in any way. 

It is fully acknowledged in the QAF that Ontario public universities have 
the primary responsibility for academic standards, quality assurance, and program 
improvement, and maintain their autonomy for priority setting and decision 
making. Abiding by the QAF, all 24 universities are committed to developing their 
own IQAPs that are consistent not only with their own institutional missions and 
visions but also with the four protocols under the QAF — new program approvals, 
expedited approvals of program modifications, cyclical program reviews, and an 
audit process. Accountability is ensured in that the IQAP of each university has 
to be ratified by the Quality Council before its implementation. All universities 
are also held accountable to a set of province-wide Degree Level Expectations 
(DLEs) [ 3] , the degree profiles for graduates from Ontario universities. The QAF 
aims to strike a balance between the needs for accountability and continuous 
program improvements as it emphasizes “quality assurance that produces quality 

[2] The UPRAC reviewed policies, procedures, and practices, rather than assessed individual programs. 

[3] There are two separate DLEs: one for undergraduate programs and the other for graduate programs. The 
DLEs are represented by a matrix that describes six categories of broadly defined learning outcomes for 
graduates of bachelor’s degrees, master’s degrees, and doctoral degrees. 
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enhancement ” (Ontario Universities Council on Quality Assurance [OUCQA], 
2014, p. 1). 

The QAF document presents itself in an illustrative and well-articulated 
way, with flow charts used to demonstrate the whole processes of new program 
development and cyclical program reviews. A cyclical review typically starts with 
a self-study and is then followed by external peer reviews before proceeding to 
the stages of institutional evaluation and formulation of implementation plans. 
The executive summary and the associated implementation plan are required to 
be posted on the institution’s website and a copy provided to the Quality Council. 
In the case of institutional auditing, the audited university is provided with an 
opportunity to comment on the audit report before the report is finalized; the 
summary of the report is publicly available on the website of the Quality Council. [4] 5 
The conclusions of the audit report fall into two categories: suggestions (desirable 
areas universities can address voluntarily) and recommendations (areas that 
universities must address). Thus, the results are not punitive. 

3.2 The OCQAS Framework for the College Sector 

The Ontario College Quality Assurance Service (OCQAS) for the college 
sector has two functional components: (1) The Credential Validation Service 
(CVS), which provides program-level quality assurance through validation of 
programs of instruction; and (2) the Program Quality Assurance Process Audit 
(PQAPA), which is an institutional-level mechanism involving a 5-year cyclical 
audit of each college’s program quality assurance processes. The new institutional 
accreditation process that started in fall 2015 is built upon the existing CVS and 
PQAPA functions. 

The CVS and PQAPA processes did not begin their operations at the same 
time. The CVS is mandated by a government policy — the Minister’s Binding 
Policy Directive, the Framework for Programs of Instruction of 2003, of which 
the legal authority is enshrined in the legislation, the Ontario Colleges of Applied 
Arts and Technology Act 2002. The policy directive established two requirements 
of the college sector: (1) a system- wide credential validation service executed 
on the basis of the Credentials Framework/ 51 which represents the minimum 
provincial standards for an array of college credentials ranging from certificates 

[4] The summary audit reports can be found on the website of OUCQA (2015). 

[5] The Credentials Framework includes these components: scope of curriculum outcomes; breadth and depth 
(including complexity of knowledge and vocational outcomes, essential employability skills, and general 
education); typical duration for completion; admission requirements; and name of credential. 
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to graduate certificates; and (2) institutional commitment to establishing 
mechanisms for program reviews to “ensure ongoing quality, relevancy, and 
currency” (Ontario Ministry of Training, Colleges and Universities [OMTCU], 
2003, p. 5). In response to the first requirement, the CVS began its operation in 
February 2005. The establishment of the audit model (PQAPA) was the response 
of Ontario colleges to the second requirement but was more of an initiative of 
colleges themselves. The model was developed through a joint government- 
college working group between 2003 and 2005 and was tested out in a pilot 
project at five colleges in 2006 before it was approved by the Committee of 
Presidents of Ontario colleges for full implementation in January 2007. Thereby, 
quality assurance mechanisms for the college sector extended from program-level 
credential validation to institutional-level auditing. 

It is important to note that the CVS does not approve programs but validates 
them against the Credentials Framework. This happens in the context where the 
responsibility for the approval of programs has been moved from the government 
to the local Board of Governors at each college (Klassen, 2012). Thus, there 
is no need to have an external agency to approve, or to involve elaborate 
procedures. As of 2012, the CVS had processed an average of 200 proposals per 
year, approximately 80% of which were for new programs and 20% for existing 
programs that needed to change titles or credentials (Klassen, 2012). 

The PQAPA process is more elaborate than that of the CVS. Its intention 
is “designed to be a developmental activity” and seeks to “contribute to the 
continuous improvement of the educational programs of the college system” 
(Ontario College Quality Assurance Service [OCQAS], 2014, p. 20). The quality 
audits operate on a 5-year cycle. Each audit starts with a self-study within the 
college and a subsequent site visit by an audit panel. Six criteria [6] drawn from 


[6] These six criteria (OCQAS, 2014, pp. 8-9) are: 

Criterion 1 . Program-level learning outcomes for all programs of instruction are set, are consistent with the 
college mission and the programs’ intended purpose, and are appropriate for the credential offered upon 
successful completion of the program; 

Criterion 2. Admission, credit for prior learning, promotion, graduation, and other related academic policies 
support program development and student achievement of program learning outcomes; 

Criterion 3. Programs conform to the Framework for Programs of Instruction and the Credentials Framework, 
are consistent with accepted college system nomenclature / program titling principles, and maintain relevance; 
Criterion 4. Methods of program delivery and student evaluation are consistent with the program learning 
outcomes; 

Criterion 5. Human, physical, financial, and support resources to support student achievement of program 
learning outcomes are available and accessible; 

Criterion 6. Regular program quality assessment that involves faculty, students, industry representatives, 
and others as appropriate for the purpose of continual improvement is in place and happens. 
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global exemplary practices are used as the standards for both the self-study 
and the audit. The findings of the audit panel are grouped into three categories: 
commendations (areas of exemplary practices), affirmation (areas identified 
by the college itself as weaknesses), and recommendations (areas identified by 
the audit panel for improvement). The audit report includes the judgments of 
“met,” “partially met,” or “not met” against the six individual quality criteria, 
as well as the maturity level [7] of the college’s quality assurance processes. The 
draft report is forwarded to the audited college for response within 30 days, and 
the responses are then incorporated into the final report. An 18-month follow- 
up report is required of the audited college to describe how it has addressed the 
recommendations in the audit report. The executive summary of each final audit 
report is posted on the PQAPA website until the next audit is completed. 

According to the 2003 policy directive, all college programs, regardless of 
funding sources, must conform to the Credentials Framework and the Program 
Standards. Developed under the sponsorship of the Ontario government, both of 
the standards are outcome-based and crucial to the CVS and PQAPA procedures. 
The Program Standards [8] represent a provincial-level benchmark for all similar 
programs offered by different colleges, and the government began to produce 
those standards for college programs in the 1990s. 

3.3 The PEQAB Framework for Degree-Granting Programs outside 
Public Universities 

In Ontario, public universities are granted full degree-granting authority by their 
legislations; this “public monopoly” was formalized under the Degree Granting Act 
of 1983 (Skolnik, 1987). Private universities are granted restricted degree-granting 
authority by the Legislative Assembly of Ontario. Any other type of postsecondary 
institutions that would like to offer degree programs has to go through a series 
of procedures to obtain the permission from the government. The Postsecondary 
Education Quality Assessment Board (PEQAB) framework is part of this process. 


[7] There are five categories of maturity of quality assurance processes: minimal effort, reactive effort, formal 
effort, organizational effort, and mature effort (OCQAS, 2014, pp. 28-29). 

[8] A program standard consists of vocational learning outcomes, a generic skills standard (known as essential 
employability skills), and general education requirements, all which are laid out consistently with the 
Credentials Framework. Not all college programs have corresponding Program Standards; however, those 
programs that have a Program Standard must meet all the requirements of the Standard. Those college 
programs that do not have existing Program Standards must develop a program description, which has to be 
aligned with the Credentials Framework as well. The vocational learning outcomes are typically developed 
under the leadership of a project officer seconded from an Ontario college to the government, OMTCU, 
with consultations with stakeholders of the affected programs from colleges and industry. 
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The PEQAB is an advisory board to the Ontario government minister 
responsible for postsecondary education. It was created and mandated by the 
Postsecondary Education Choice and Excellence Act, 2000. [9] The legislation 
was developed after a public consultation as to whether to increase opportunities 
for degrees in Ontario (OMTCU, 2000). Essentially, the Act extends the scope of 
degree-program providers from public universities to public colleges, in-province 
private institutions, and out-of-province organizations, which are permitted to 
seek ministerial consent [10] as an additional option * [11] in order to offer programs 
leading to a degree in Ontario. Applicant organizations can apply for ministerial 
consent for new programs and for renewal of the ministerial consent for ongoing 
programs. Working with public colleges, private in-province institutions, and out- 
of-province organizations, the PEQAB reviews their applications made under 
the 2000 Act and makes recommendations to the Minister. It operates under the 
leadership of a management board, with government-appointed members from 
various professional backgrounds. 

The key component of the PEQAB process is quality assessment conducted 
by an expert review panel, which is appointed by the PEQAB. The panel 
reviews program quality and institutional capacity against PEQAB standards and 
benchmarks, and writes up a report to the PEQAB. Private organizations are also 
subject to an organization review by a separate panel. The applicant organization 
has an opportunity to submit a formal response to the assessment report. All 
the applications are posted on the PEQAB website, and interested parties from 
the public have an opportunity to provide comments by indicated deadlines. 
Applicants seeking renewal of ministerial consent need to go through a program 
self-study and a site visit by the program evaluation committee of the applying 
institution. By March 31, 2014, the PEQAB had completed the assessment of a 
total of 457 applications from 70 institutions (Postsecondary Education Quality 
Assessment Board [PEQAB], 2014a). 

The quality of proposed degree programs, whether from public or private 
organizations, is assessed against the same thirteen standards and related 

[9] The Act can be accessed from the online repository of the Government of Ontario (2015). 

[10] “Ministerial consents” are decisions of the Ontario government to permit private and/or foreign institutions 
to offer degree programs in Ontario. The practice started in 1994 when the NAFTA (the North American 
Free Trade Agreement) united Canada, the United States, and Mexico in a common market (Baker & 
Miosi, 2010). The consent is usually granted for five years, and a renewal of the consent must be sought. 

[11] Under the Degree Granting Act (1984 ~ 2001), the predecessor of the 2000 Act, an Ontario-based 
institution required an act of Legislative Assembly of Ontario in order to grant degrees, offer programs 
leading to a degree, call itself a university, or advertise using the word university (PEQAB, 2014b, p. 1). 
Thus, seeking ministerial consent becomes an additional option under the 2000 Act. 


Liu: The Quality Assurance System for Ontario Postsecondary 65 

Education: 2010-2014 

benchmarks. [12] The central one is the degree level standard, which is comprised 
of degree standards for graduates of bachelor’s, master’s, and doctoral degree 
programs in Ontario. These standards are the same as those ones for degree 
programs in the Ontario Qualifications Framework/ 131 and overlap with the 
DLEs used in the Quality Assurance Framework for the university sector. Thus, a 
common set of degree standards is used by the Quality Council and the PEQAB. 

3.4 Summary 

It can be argued that the Ontario system of quality assurance for 
postsecondary education is heavily policy-driven and buttressed by the work 
of the three external quality assurance agencies as the pillars — the Quality 
Council, the OCQAS, and the PEQAB. The related policies and processes are 
summarized in the following two tables from different points of view. Table 1 
shows a comparison of the three quality assurance frameworks in terms of the 
agency, initiating legislation or policy, institutional stakeholders, mechanisms, 
approaches, processes, and standards. 

Table 2 shows how various quality assurance mechanisms correspond to the 
different types of postsecondary institutions within the Ontario system. The table 
suggests that academic programs offered by publicly assisted Ontario universities 
and colleges, whether new or ongoing and inclusive of both degree and non- 
degree programs, are subject to certain formalized quality assurance mechanisms. 
For other types of institutions, some form of external assurance mechanisms is 
in place too. Thus, the Ontario quality assurance system appears generally well 
balanced among sectors. 

4. Evaluation of the Ontario Quality Assurance System (2010 - 
2014) 

The operations and processes represented by the three quality assurance 
frameworks as discussed above speak to the following four features of the Ontario 
quality assurance system. Some of them are informed by the general model for 
quality assessment (van Vught & Westerheijden, 1994) and the Phase Model of 
quality assurance systems development (Jeliazkova & Westerheijden, 2002). 


[12] The thirteen standards are degree level; admission, promotion and graduation; program content; program 
delivery; capacity to deliver; credential recognition; regulation and accreditation; nomenclature; program 
evaluation; academic freedom and integrity; student protection; economic need; and non-duplication 
(PEQAB, 2014b, pp. 14-15). 

[13] The Ontario Qualifications Framework lays out a spectrum of qualification descriptions and standards for 
all non-religious certificate, diploma, and degree programs offered under the auspices of Ontario. It can be 
found on the website of OMTCU (2015). 
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Table 1 . Comparison of the Three External Quality Assurance Frameworks in Ontario 


Quality Assurance OCQAS Framework PEQAB Framework 


External quality 
assurance agencies 

OUCQA, or the Quality 
Council 

OCQAS 

PEQAB 

Legislations or 
policies 

Quality Assurance 
Framework, 2010 

The Minister’s Binding 
Policy Directive: 
Framework for Programs 
of Instruction, 2003 

Postsecondary Education 
Choice and Excellence 
Act, 2000 

Types of 

institutional 

stakeholders 

Ontario (public) 
universities 

Ontario (public) colleges 

Ontario public colleges; 
Private career colleges; 
Organizations (private 
or public) based in other 
jurisdictions 

External quality 

assurance 

mechanisms 

New program appraisal 
(program-level); 
Institutional quality 
assurance process audit 
(institutional-level) 

Credential validation 
(program-level); 
Program quality 
assurance process audit 
(institutional-level) 

Degree program quality 
assessment (program- 
level) for the purpose of 
granting or renewal of 
ministerial consent; 
Organization review for 
private organizations 
(organizational-level) 

Approaches 

Learning outcomes 
based; 

Self-regulation through 

auditing; 

Developmental 

Learning outcomes 
based; 

CVS: government- 
mandated; 

PQAPA: Self-regulation 
through auditing; 
Developmental 

Learning outcomes 
based; 

Direct assessment; 
Government-mandated 

Processes 

involved 

Self-study; 

Peer review, with site 
visit; 

Reporting, providing 
opportunities for 
institutions to respond 

Self-study; 

Peer review, with site 
visit; 

Reporting, providing 
opportunities for 
institutions to respond 

Self-study; 

Peer review, with site 
visit; 

Reporting, providing 
opportunities for 
institutions to respond 

Learning 

outcomes 

standards 

Degree-Level 
Expectations (for 
undergraduate and 
graduate programs) 

Credentials Framework; 
Learning outcomes in 
pre-defined Program 
Standards 

Four sets of degree 
standards within the 
Ontario Qualifications 
Framework 

Other standards 

Evaluation criteria for 
new program proposals 
and program reviews 

Six criteria for exemplary 
quality assurance 
processes 

Thirteen standards and 
related benchmarks for 
quality programs 


Source: This study. 
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Table 2. Association between the Quality Assurance System and the Postsecondary 


Education System in Ontario 

Types of institution 

Program status 

External quality assurance mechanisms, including 
administering agencies 

Public universities 

New 

Ongoing 

New program appraisal by the Quality Council 
(Appraisal Committee) 

Institutional audit by the Quality Council (Audit 
Committee) 


Regular (non-degree) 
programs -- new 

Credential validation by OCQAS (CVS) 


Regular (non-degree) 
programs - ongoing 

Program quality assurance process audit by 
OCQAS (PQAPA), including a self-study 
requirement and an audit 

Public colleges 

Degree programs — 
new 

Granting of ministerial consent, subject to 
recommendations by PEQAB after an external 
review by the PEQAB-appointed Quality 
Assessment Panel 


Degree programs — 
ongoing 

Renewal of ministerial consent, subject to 
recommendations by PEQAB after an external 
review by the PEQAB-appointed Quality 
Assessment Panel 


Granting or renewal of ministerial consent, 
Other (private institutions N , . subject to recommendations by PEQAB after an 

in Ontario cmri mit_r»f_ ew an on g° m g external review by the PEQAB -appointed Quality 


in Ontario, and out-of- 
province organizations) 


degree programs 


Assessment Panel and Organization Review 
Committee (for private organizations) 


Source: This study. 


4.1 Formalized Quality Assurance Mechanisms with Diversity 

In the Ontario quality assurance system, as shown in Table 1, all three quality 
assurance processes feature arms-length agencies, self-studies at postsecondary 
institutions, peer reviews with site visits, and high transparency in reporting with an 
opportunity for the evaluated institution or program to comment on the report. For 
ongoing programs at both universities and colleges, no direct relationship has been 
set up between the outcomes of the audit reports and any funding decisions. 
However, there are funding implications for new programs to be offered by 
public universities and colleges because funding approval is de facto approval 
of the program (Leyton-Brown, 2005). A new college program that fails credential 
validation by the CVS and a new university program that fails the approval by the 
Quality Council are not in good positions to seek governmental funding. In the 
actual practice, failures have rarely occurred partly due to the fact that the two 
agencies provide considerable amounts of support to applicants to bring their 
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proposed programs up to the standards. All these features attest to the elements in 
van Vught and Westerheij den’s (1994) general model for quality assessment. 

Table 1 also shows that although the three frameworks entail similar 
procedures, they involve quite diverse quality assurance mechanisms. Those 
mechanisms incorporate the institutional quality that involves institutional 
infrastructure and governance, as well as the program quality that involves the 
curriculum and outcomes of the program (Leyton-Brown, 2005). In addition to 
the internationally accepted practices of program appraisal and quality audits, 
the Ontario system also utilizes credential validation and granting or renewal of 
ministerial consent as quality assurance mechanisms. Credential validation is 
employed to honour the autonomy of Ontario colleges in managing their own 
academic programming. Use of ministerial consent is a practice that was carried 
forward from a previous governmental action for similar scenarios. They appear 
to be unique to Ontario. 

Further, the themes of quality control, quality assurance, quality assessment, 
and quality improvement have all emerged in the Ontario quality assurance 
system. The government exerts strict control of final degree-granting authority 
by enforcing a ministerial consent mechanism. The PQAPA process makes 
a deliberate distinction between quality assurance and quality assessment by 
defining quality assessment as “the mechanism or procedures used to determine 
the extent to which quality exists” and quality assurance as “the mechanism or 
procedures used to assure or measure the level or existence of quality” (OCQAS, 
2014, p. 5). In other words, the assumption behind quality assurance is that 
quality exists and the task of an external agency is to verify that the existing 
process has made an impact on academic programs. Further, all three frameworks 
take stock of the importance of continuous improvement in quality, rather than 
being satisfied with mere quality assurance. While the PEQAB process is more 
accountability-driven, continuous improvement is part of its benchmarks of the 
program evaluation standard. In practice, the QAF and the PQAPA processes 
are primarily formative and developmental in nature, and both require an 
implementation plan and a follow-up report to ensure continuous improvement. 

4.2 The Arms-Length Tripartite Relationship with a Distinction 
between Self-Regulation and Government Regulation 

All three quality assurance agencies maintain an arms-length relationship 
with postsecondary institutions and the government, with subtle but important 
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differences. The PEQAB is the only government-appointed agency of the three, 
but it independently manages the quality assessment process. The Quality 
Council is what Baker and Miosi (2010) called “university consortium agency.” 
It was created as part of the Council of Ontario Universities, a body on behalf 
of the university sector that promotes cooperation among public universities, 
and between them and the provincial government (Council of Ontario 
Universities [COU], 2012) and is funded by Ontario universities and governed 
by representatives almost entirely from Ontario universities. The QAF states that 
it “operates at arm’s length from universities and the government to ensure its 
independence” (OUCQA, 2014, p. 2). The OCQAS operates independently of 
the government and of any Ontario college, and its management board members 
are mostly representatives who are affiliated or associated with the Ontario 
college sector. Like the Quality Council, the service at OCQAS is “funded by, 
and responsible to, the 24 member colleges, and not to the government” and it is 
“owned and operated by the colleges,” as a past executive director emphasized 
(Klassen, 2012, p. 6). 

Related to the different relationships with the government, the QAF process 
administered by the Quality Council and the PQAPA process under the OCQAS 
framework employ a self-regulation approach whereas the PEQAB framework 
essentially adopts a government regulation approach. Unlike the self-regulation 
of internal quality assurance executed by individual institutions, the system-level 
self-regulation is administered collectively (Baker & Mioni, 2010) — through the 
Quality Council for the university sector and through the OCQAS for the college 
sector. Perhaps because of the nature of self-regulation in the QAF and PQAPA 
processes, the work of the Quality Council and the OCQAS is partly consultative 
and collaborative whereas the PEQAB does not have a consultation role to play. 

The nature of self-regulation is exhibited by how the mechanisms were 
created as well as how they operate in actuality. The self-regulation approach to 
the OCQAS framework is well documented in the PQAPA orientation manual, 
which clearly states that the PQAPA is “a self-regulating process at the system 
level” (OCQAS, 2014, p. 5). The self-regulation approach is also reflected in 
the choice of using quality audits as the primary external quality assurance 
mechanism. As the PQAPA manual explains in the footnote, audit is used to 
mean that PQAPA “reviews processes, validates the college’s Program Quality 
Assurance record, and verifies that the process has the impact of improving 
programs” (OCQAS, 2014, p. 5). Thus, the self-regulatory function of the 
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OCQAS is fulfilled by the quality audit under PQAPA, which assumes the 
existence of quality and internal quality assurance mechanisms inside Ontario 
colleges and hence intends to find the evidence to verify this assumption 
(Klassen, 2012). As a past chair of the Management Board commented regarding 
the creation of the two components of the OCQAS, the government’s mandate 
created CVS but it was the colleges themselves that created the PQAPA at the 
time when the government intended to reduce its involvement in managing 
college programs but grant Ontario colleges greater autonomy in program 
approval and development; therefore, establishing a self-regulating process for 
the whole college system was considered in the interests of both the government 
and the colleges (interview, November 21, 2013). OCQAS ’s ownership by the 
college sector was found not to inhibit the fulfillment of its functions in any way, 
according to the report of an external PQAPA review (Massy, 2006). 

Similarly, the QAF is a self-regulatory policy although this is not made 
explicit in its document. Self-regulation began with its predecessor, the OCGS 
appraisal process, where public universities were bound to not implementing any 
new graduate programs until they were appraised by OCGS and found to be of 
good quality (Leyton-Brown, 2005). A key member on the QAF development 
taskforce stated that it was the Council of Ontario Universities who approved 
the creation of the Quality Council; in that sense, the Council was created by the 
Ontario universities themselves. He further explained, “So it is self-regulatory 
system, not imposed by the government but self-imposed by the system. It is a 
bit unusual because in many places, it is a government body that requires these 
things. Ontario is a little bit different — the government has given the authority 
to the Council of Ontario Universities, or previously the OCGS or the UPRAC 
through OCAV.... The current mechanism is a continuation and a further 
formalization” (interview, October 30, 2013). A Quality Council staff member 
also verified this by stating that “it is a self-regulatory system.... When you self- 
regulate, because you don’t want government coming in. The universities do 
not want the government of Ontario to do this work. I think their intention was 
to make it more rigorous than anybody else so that it can’t be criticized, it can’t 
be accused of being weak.... It’s a smart strategy for the universities to have” 
(interview, March 27, 2014). Further, the QAF requires all Ontario universities 
to develop their own institutional quality assurance processes, so that they take 
ownership and commit themselves to implementing their own mechanisms; they 
also voluntarily agree that this whole process is overseen by the Quality Council. 
This is a scheme of “self-regulation with rules” (Lang, 2015). The use of the self- 
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regulation approach suggests that the Ontario quality assurance system is in the 
relatively mature Phase 4, according to Jeliazkova and Westerheijden’s (2002) 
model. 

4.3 Mixed Phases of Quality Assurance Development and the Maturing 
Process 

When examined in light of Jeliazkova and Westerheijden’s (2002) Phase 
Model, various quality issues could be captured by the problem statements in 
the model, albeit not precisely. Before 2000, there was high trust in the quality 
of programs at Ontario universities and colleges, instead of serious doubts; in 
addition, the primary role of quality assurance was not to identify sub-standard 
educational programs. However and legitimately, there were serious concerns 
about the quality of degree programs offered by emergent degree providers, 
such as public colleges and out-of-province organizations. Some level of quality 
culture existed in the university sector due to the previous OCGS and UPRAC 
mechanisms while some were concerned that the college sector was over- 
supervised by the government. 

The three quality assurance frameworks were established to address those 
quality issues. The PEQBA framework, as part of the ministerial consent process, 
was created to address the concern about degree-granting issues and assure that 
the degree programs offered by those emergent degree providers follow the same 
standards as those programs at public universities; therefore, its assessment is 
mostly summative. The self-regulatory quality audits that QAF and OCQAS 
frameworks have adopted are to stimulate the institutional capacity for quality 
assurance and continuous quality improvement; thus, the processes are mostly 
formative. Experience elsewhere shows that use of academic audits as a quality 
assurance mechanism can contribute to achieving the dual goals of accountability 
and improvement (Dill, 2000). The Ontario experience seems to affirm that 
accountability and improvement can be balanced through quality audits. As 
such, the quality assurance solutions appear to be at different phases of the Phase 
Model, depending on the nature of the problems. 

Further, under the QAF and OCQAS frameworks, postsecondary institutions 
are not dictated by the government to solve quality assurance issues but are 
guided to build their own capacity for quality management. The responsibility for 
quality assurance is mostly in the hands of postsecondary institutions themselves. 
The expanded institutional autonomy for quality assurance reflects a level of 
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maturity in managing quality issues on the part of postsecondary institutions. A 
past OCQAS director used an analogy to describe this maturing process: “It is 
like kids. When they are little, they are regulated by their parents. At some point, 
you hope they become self-regulated. They learn how to make good decisions 
and look after themselves. It is the same for the evolution of the college system. 
It was time, particularly with the devolution of responsibility and autonomy, 
that colleges could say ‘We can do this ourselves, we can make sure that our 
programs are meeting the standards, we have a quality system and we don’t 
need the government to be telling us about it’” (interview, June 20, 2013). This 
maturing process reflects the evolution from Phase 1 to Phase 4 in Jeliazkova and 
Westerheij den’s (2002) Model. 

An indicator of the maturing process is the shifted focus of quality 
assurance from external to internal dynamics; in the meantime, this shift gives 
rise to standardization. The implementation of the external quality assurance 
mechanisms helps bring forth organizational changes within postsecondary 
institutions by affecting their internal quality management. Each Ontario 
university has to establish its own quality assurance process, known as the IQAP, 
which more or less follows a standardized institutional process prescribed by the 
QAF. Within the college sector, internal quality assurance mechanisms become 
more alike than before as a result of their being evaluated against the same six 
quality criteria set by the PQAPA. The quality management of degree programs, 
at public and private organizations, has to include a program evaluation 
committee, thus taking on a similar governance structure for quality assurance, 
as a result of the requirements made by PEQAB to those programs seeking 
ministerial consent. Academic programs at Ontario colleges and universities 
have to abide by the learning outcomes and academic standards required by 
the three quality assurance frameworks. As such, the external quality assurance 
mechanisms are steering changes in many areas within postsecondary institutions 
in Ontario, and the internal quality assurance processes are becoming more 
consistent and standardized. The push toward standardization is often found 
problematic by university faculty. 

4.4 Gaps in the System 

A significant gap in the Ontario system is inadequacy in quality assurance 
mechanisms for private educational providers. Although those private providers 
that offer programs leading to non-degree credentials are, in principle, subjected 
to the same Credentials Framework as public colleges, they do not have to go 


Liu: The Quality Assurance System for Ontario Postsecondary 73 

Education: 2010-2014 

through the OCQAS processes, thus falling short of external quality assurance. 
Although degree program offering is under the tight control of the government, 
the ministerial consent mechanism applies only to the beginning stage of a new 
program, and how the program performs later on is mainly left to the internal 
mechanism at individual institutions. 

Another deficiency lies in the lack of clarity of some key areas in the 
Quality Assurance Framework. As alluded to earlier, it is not clear enough in 
the QAF and its guide that the framework is essentially self-regulatory, rather 
than being imposed by the government. When this distinction is not clarified, 
misunderstandings have occurred and challenges have arisen when the QAF is 
implemented. A program evaluation committee member revealed his incorrect 
assumption about the QAF and the frustration of some of his colleagues at his 
university by saying “It is not our committee that says ‘You have to do this, this, 
and this.’ Some of the guidelines we have to address are being dictated to us by 
the province. They want these things to be addressed in the programs. We are 
really just following the law. People don’t understand why we have to do this. 
[We say] ‘Well, if you don’t get your program evaluated, you can’t deliver your 
program’” (interview, April 28, 2014). A Quality Council staff shared how she 
addressed the frustration by saying, “I think many universities have struggled 
with certain parts of it and I often say to them, ‘Well, it is your policy.’ Especially 
when you have lots of turnover in personnel in universities, sometimes people 
will express a frustration with it. I will remind them that it is something they 
created” (interview, March 27, 2014). Similar frustration and difficulty can 
be diluted or avoided when the nature of the QAF is made clear to those who 
implement the framework. 

Finally, the three quality assurance frameworks were developed on relatively 
independent paths, without a master design. The operations of the three quality 
assurance agencies remain quite independent and uncoordinated. There is a lack 
of formal venues in which the three agencies could share experiences and lessons 
learned. Although the DLEs in the QAF and the degree standards in the PEQAB 
framework are consistent and overlapping, disagreement exists in terms of the 
authorship of those standards. This lack of coordination may be a vestige of the 
prior separately operating arrangements of quality assurance and result partly 
from a lack of totality in Canadian postsecondary education itself (Jones, 1997) or 
the absence of a systematic approach to policy-making in Ontario (Royce, 1998). 
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5. Interpretations and Conclusions 

The emergence of the Ontario quality assurance system appears to reflect a 
path trajectory of governance structure development of the Ontario postsecondary 
education system as well as variations to address emerging issues. The practice 
of setting up an advisory body between the government and postsecondary 
institutions started when the first governmental department was established in the 
1960s. As Jones (1997) documented, in 1964 when the Ontario government, for 
the first time, created a governmental department managing university affairs, 
they also created a neutral advisory committee, which was designed to play a 
buffer role, rather than act as a direct appendage of the government. To address 
degree-granting issues with emergent program providers, the PEQAB becomes 
another body that plays such a buffer role. 

Historically, Ontario universities have a high level of institutional autonomy. 
Maintaining high autonomy at universities is always an important consideration 
and direct control of the university sector has never been an option or preference 
for the government (Jones, 1997; Royce, 1998) although university autonomy was 
challenged from time to time by some government initiatives (Shanahan, Jones, 
Fisher, & Rubenson, 2014). Establishment of a self-regulatory quality assurance 
mechanism through an intermediary body (the Council of Ontario Universities) 
appears to be an optimal solution given the governance tradition in Ontario and 
the need to balance accountability and institutional autonomy. 

The college sector has traditionally been far less autonomous than the 
university sector in Canada. However, as the government moved in the direction 
of deregulation in order to encourage competition and reduce the governmental 
share of funding, Ontario colleges were encouraged to become more autonomous. 
This tendency emerged in the 1990s when the province experienced financial 
constraints (Jones, 1997). The 2003 policy directive, which granted colleges’ 
boards of governors the authority to approve programs, was a policy movement 
toward greater institutional autonomy of Ontario colleges. Thus, Ontario colleges 
gained responsibility and flexibility, and the relationship between the colleges and 
the government shifted “from a command and control model, to an accountable 
for results model” (Charles, 2011, p. 278). The PQAPA process represents another 
important mechanism that helps cultivate and formalize the internal quality 
assurance processes within a sector- wide framework. 

It can also be argued that the Ontario quality assurance system demonstrates 
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a result of the interactions between the local and the global. The high institutional 
autonomy and the limited policy capacity of the government to intervene have, 
to a large extent, determined the quality assurance arrangements in Canadian 
universities (Weinrib & Jones, 2014), and influenced the formulation of quality 
assurance processes in Ontario. The Ontario quality assurance system is a 
concrete example that exhibits those Canadian characteristics and the sector- 
specific conditions as presented above. On the other hand, globalization has 
inevitably exerted considerable impact on the three Ontario quality assurance 
frameworks. All three were developed under the global trend of strengthening 
accountability and continuous improvement in postsecondary education. 
Aspirations to align with “international best practices” are manifested in the 
documents of all three frameworks. Use of the learning outcomes approach and 
the tendency of standardization in processes are also evidence of global influence. 

The two conceptual models (Jeliazkova & Westerheijden, 2002; van 
Vught & Westerheijden, 1994) used in this paper are useful analytical tools for 
examining the characteristics of a quality assurance system. However, the Ontario 
case shows that certain distinctive features of a particular system may be hard to 
schematize. One single system can incorporate quality assurance mechanisms that 
address problems across different phases at the same time, as the Ontario system 
does. Thus, the Phase Model, though helpful to better understand the development 
of quality assurance schemes, may be oversimplified to represent the complexity 
of a quality assurance system. 

The landscape of quality assurance for Ontario postsecondary education 
has changed significantly since 2000. Before that, the programming at Ontario 
colleges was overseen by the government, quality assurance of university 
undergraduate and graduate programs was subject to completely separate 
mechanisms, and there was no formal quality assurance procedure for degree 
education provided outside public universities. Currently, a comprehensive and 
relatively mature quality assurance system with diverse but rigorous mechanisms 
is in place, consisting of three quality assurance frameworks that have evolved on 
independent paths. The three frameworks embody careful deliberations of needs 
for accountability and continuous improvement and the tripartite relationship 
among the postsecondary institutions, the government, and the quality assurance 
agencies. They represent important accomplishments in strengthening quality 
assurance mechanisms in Ontario, although those mechanisms remain as 
uncoordinated as before. 
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In 2015, a new mandate of institutional accreditation for Ontario’s college 
sector is taking effect under the OCQAS framework. This reflects another 
impact of globalization. A brand new chapter of quality assurance is unfolding 
for postsecondary education in Canada, where no institutional accreditation 
previously existed in history. The Ontario quality assurance system continues to 
evolve and will take on additional meanings. 
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